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PREFACE 


This  report  le  concerned  with  the  influence  of  display  needle 
sensitivity  on  the  accuracy  with  which  pilots  can  fly 
nonprecision  approaches.  Twelve  private  pilots  flew  an 
Instrumented  single  engine  airplane  on  144  approaches  at  six 
different  sensitivity  levels.  Increases  In  needle  sensitivity 
produced  decreases  In  racking  error,  but  only  at  the  cost  of 
increases  In  pilot  workload. 

This  report  was  prepared  by  the  Operator  Performance  and  Safety 
Analysis  Division  of  the  Office  of  Research  and  Analysis  at  the 
Volpe  National  Transportation  Systems  Center  (VNTSC)  for  the 
FAA's  Research  and  Development  Service. 

The  research  was  directed  by  M.  Stephen  Huntley  Jr.  Data  for  the 
report  were  collected  by  Christopher  J.  Rourke  and  analyzed  by 
Robert  M.  Disario. 
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EXECUTIVE  SUMMARY 


This  study  was  conducted  to  determine  the  influence  of 
Course  Deviation  Indicator  (GDI)  sensitivity  on  pilot  tracking 
error  during  nonprecision  approaches.  Twelve  pilots  flew  an 
instrumented  single-engine  airplane  on  144  approaches  at  six 
different  levels  of  GDI  sensitivity.  The  sensitivities  ranged 
from  15,190  feet  (2.5  nautical  miles)  to  475  feet  (0.08  nautical 
miles)  for  a  full-scale  deflection.  Increases  in  sensitivity  of 
this  magnitude  decreased  crosstrack  Root  Mean  Square  (RMS)  error 
from  an  average  of  0.22  to  0.04  nautical  miles.  Magnitude  of  the 
error  and  the  influence  of  sensitivity  on  that  magnitude  were 
affected  by  distance  from  the  missed  approach  point.  Pilots 
reported  that  increases  in  sensitivity  increased  their  workload 
and  changed  their  distribution  of  attention  among  the  aircraft 
instruments  used  for  navigation  and  directional  control. 
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1. 


INTRODUCTION 


1 . 1  BACKGROUND  AND  OBJECTIVES 

The  FAA  approves  instrument  approach  procedures  at  all 
locations  where  the  United  States  has  jurisdiction  over  flight 
procedures  in  terminal  areas.  These  procedures  include  the  track 
and  altitude  profile  that  aircraft  will  be  required  to  fly,  as 
well  as  the  depth,  width,  and  horizontal  dimensions  of  the 
involved  airspace  that  must  be  guaranteed  obstruction-free.  The 
width  of  the  path  that  must  be  cleared  is  determined  by  measuring 
sources  of  navigation  system  error.  One  of  these  is  "Flight 
Technical  Error"  (FTE)  or  "crosstrack  error,"  which  refers  to  the 
accuracy  with  which  the  pilot  controls  the  aircraft.  This  is 
measured  by  the  discrepancy  between  the  indicated  command  on  the 
display  and  the  actual  aircraft  position.  The  smaller  the  FTE, 
the  narrower  the  path  width  that  must  be  clear  of  obstruction. 

The  display  often  used  to  indicate  the  aircraft's  position 
relative  to  the  desired  track  is  called  the  course  deviation 
indicator  (GDI) .  The  principal  objective  of  this  study  was  to 
determine  the  influence  of  GDI  sensitivity  on  FTE  (an  increase  in 
GDI  sensitivity  results  in  greater  deflections  of  the  GDI  needle 
for  a  given  displacement  of  the  aircraft  from  the  desired  track) . 
Other  study  objectives  included  determining  the  influence  of  GDI 
sensitivity  on  pilot  workload  and  aircraft  handling. 


1 . 2  METHODOLOGY 

Data  on  flight  performance  was  collected  from  twelve 
instrument-rated  pilots  who  flew  nonprecision  instrument 
approaches  at  six  different  levels  of  GDI  sensitivity  into  a 
local,  uncontrolled  airport.  The  approaches  were  flown  in  an 
instrumented  Piper  Archer  airplane,  equipped  with  a  LORAN-G 
receiver.  Each  pilot  flew  all  approaches  wearing  a  hood  and 
provided  estimates  of  pilot  workload  during  each  approach. 

Safety  pilots  provided  estimates  of  pilot  effort. 

The  remainder  of  this  section  describes  the  characteristics 
of  the  pilots,  the  aircraft  and  instrumentation  used,  and  the 
study  procedures. 


1.2.1  Pilot  Characteristics 

The  twelve  pilots  who  participated  in  this  study  were 
volunteers  who  responded  to  a  sign-up  sheet  posted  at  the  Minute 
Man  Airfield  in  Stow,  Massachusetts.  They  normally  flew  out  of 
Minute  Man  and  were  familiar  with  the  Piper  Archer  airplane.  All 
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volunteers  were  male,  instrument-rated,  and  Instrument  Flight 
Rule  (IFR)  current.  The  group  characteristics  included: 


o 


o 


Range  of  ages  for  pilots;  20  years  to  51  years 
Mean;  36  years 

Range  of  total  flight  time;  325  hours  to  2,900  hours 
Mean;  951  hours 


o  Range  of  total  instrument  time;  46  hours  to  600  hours 
Mean:  146  hours 


1.2.2  Aircraft  and  Instrumentation 

All  data  collection  flights  were  made  in  a  Piper  Archer. 

This  light,  single-engine  airplane  with  a  fixed  gear  was  selected 
due  to  its  simplicity  and  because  it  is  familiar  to  many  pilots. 

In  addition  to  being  fully  IFR-equipped ,  the  airplane 
contained  a  Northstar  Ml  LORAN-C  receiver  and  a  second  set  of 
airplane  instruments.  The  LORAN-C  and  the  duplicate  aircraft 
instruments  were  connected  to  a  minicomputer  used  for  data 
recording  in  flight. 

The  Ml  LORAN-C  receiver  is  a  standard,  commercially 
available  unit  that  was  modified  for  this  experiment  by  Northstar 
so  that  the  following  GDI  sensitivities  could  be  selected  in 
flight  for  any  approach: 

o  1/64  nautical  miles  (nm)  per  dot  and  475  feet  full 

scale,  which  is  equivalent  to  Instrument  Landing  System 
(ILS)  at  middle  marker 

o  1/32  nm  per  dot  and  950  ft.  full  scale 

o  1/16  nm  per  dot  and  1,900  ft.  full  scale 

o  1/8  nm  per  dot  and  3,800  ft.  full  scale 

o  1/4  nm  per  dec  and  7,600  ft.  full  scale 

o  1/2  nm  pe"  dot  and  15,190  ft.  (2,5  nm)  full  scale 

Information  output  from  the  LORAN-C  included  ground  speed, 
distance  to  the  next  waypoint,  the  name  of  the  next  waypoint,  and 
crosstrack  error. 

The  second  set  of  instruments,  including  directional  gyro, 
attitude  indicator,  altimeter,  and  turn-and-sl ip  indicator,  were 
mounted  together  with  a  vacuum  pump  in  a  12-inch  square  aluminum 
box  located  behind  the  pilot’s  seat. 
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1.2.3  Subjective  Performance  Measures 


Subjective  performance  measures  included  estimates  of  pilot 
performance  made  by  the  pilot  and  the  safety  pilot  during  each 
instrument  approach.  At  1,  1.5,  and  2.5  minutes  inbound  from  the 
final  approach  fix,  the  pilot  gave  the  "experimenter"  (equipment 
operator)  a  verbal  estimate  of  his  present  workload. 

At  the  missed  approach  point,  the  safety  pilot  recorded  his 
subjective  estimate  of  how  hard  he  thought  the  pilot  worked 
during  the  approach.  Both  estimates  were  made  on  a  seven-point 
scale,  with  "seven"  indicating  very  high  workload  or  effort. 


1.2.4  Setup 

Each  pilot  flew  all  approaches  wearing  a  hood  and,  when 
cued,  estimated  his  workload.  A  safety  pilot  operated  the  LORAN, 
monitored  the  safety  of  the  flight,  and  provided  estimates  of 
pilot  effort  after  each  approach.  A  technician  (also  called  the 
"experimenter"),  in  the  rear  of  the  airplane,  operated  the  data 
recording  equipment  and  cued  the  pilot  three  times  during  the 
approach  for  workload  estimates. 

The  twelve  instrument-rated  pilots  were  divided  into  two 
groups.  Six  of  these  pilots  constituted  a  high  flying-time  group 
(100  hours  or  above);  the  other  six  constituted  a  low  flying-time 
group  (below  100  hours) .  These  flying  times  were  based  on  total 
instrum.ent  tine. 

Within  each  group,  the  six  men  were  further  divided  into 
groups  of  three: 

Number  of  pilots  (high)  Number  of  pilots  (low) 

Group  A3  3 

Group  B  3  3 


Groups  A  and  B  refer  to  GDI  sensitivity  levels  as  follows: 

Group  P  1/2  nm  per  dot 
1/8  nm  per  dot 
1/32  nm  per  dot 

Group  B  1/4  nm  per  dot 
1/16  nm  per  dot 
1/64  nm  per  dot 
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The  order  in  v;hich  the  pilots  flew  the  three  sensitivity 
levels  was  counterbalanced  within  each  group  of  six  pilots  us^'ng 
the  following  sequences,  with  one  sequence  being  assigned  to  a 
single  pilot  in  the  group  (sequence  reads  from  left  to  right) : 

123  312  231  321  132  213 


Each  pilot  flew  seven  approaches  each  on  two  seperate  days 
for  a  total  of  168  nonprecision  instruinent  approaches.  The  order 
in  which  he  got  the  three  sensitivity  conditions  was  the  sane  on 
each  day. 

The  first  approach  of  the  day  was  always  a  practice  approach 
at  1/4  nautical  mile  per  dot.  This  value  was  selected  because  it 
is  a  common  standard  setting  used  with  LORAN  systems  for 
operations  within  terminal  areas.  Data  was  recorded  during  the 
next  six  approaches.  The  pilot  made  two  consecutive  approaches 
at  each  of  the  three  sensitivity  levels  assigned  to  him,  in  the 
assigned  test  order.  The  seven  approaches  took  about  two  hours. 


1.2.5  Procedure 


Before  each  day's  flight,  the  pilot  was  verbally  briefed  and 
given  written  material  that  described  the  test  procedures.  He 
was  further  provided  with  the  appropriate  sectional  chart  and 
approach  f late  (Figures  1-1,  1-2,  and  1-3).  The  pilot  was 
encouT-ao''=^  t  ■>  rake  nctuo  on  the  plate  if  desired.  In  addition, 
he  •*  "> !  1  '"ow  th'-  seven -point  workload  scale  was  to  be  used: 

o  A  "1"  on  uh3  S',  ale  represented  very  low  workload.  It 
indicated  that  all  phases  of  the  approach  could  easily 
be  accomplished  and  that  there  was  time  to  spare  to 
attend  to  other  aspects  of  the  flight. 

o  A  "7"  on  the  scale  represented  very  high  workload.  It 
indicated  that  there  was  insufficient  time  to  attend  to 
all  of  the  approach  procedures  and  that  no  time  could 
be  spared  for  planning,  or  for  unanticipated  events. 

The  pilot  was  also  given  a  description  of  how  the  LORAN-C 
operated,  so  that  the  automatic  waypoint  sequencing  in  the  LORAN- 
C's  flight  plan  mode  was  understood.  Finally,  the  volunteer  v;as 
reminded  of  the  importance  of  keeping  the  GDI  needle  as  close  to 
the  center  as  possible  while  flying  the  approach  procedure, 
especially  at  the  course  intercept. 

All  test  flights  were  flown  between  Minute  Man  Airfield  in 
Stov/  and  the  Gardner  Municipal  Airport  in  Gardner,  a  distance  of 
about  25  miles  (Figure  1-1).  The  flight  from  Stow  to  Gardner  and 
thc!  seven  approaches  took  .nore  than  two  hours  of  continuous 
flying. 
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FIGURE  11 


MINUTE  MAN  AND  GARDNER  AIRPORT‘S  AND  ORIENTATION  OF 
INSTRUMENT  APPROACH  TO  RUNWAY  18  (GARDNER) 


FIGURE  1-2 


INSTRUMENT  PROCEDURE  CHART  FOR  RUNWAY  18 
APPROACHES 
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FIGURE  13.  INSTRUMENT  PROCEDURE  CHART  FOR  RUNWAY  36 
APPROACHES 


Depending  on  the  prevailing  winds,  the  pilot  flew  one  of  tv;o 
possible  approach  courses  for  all  seven  approaches  of  the  day 
(see  Figures  1-2  and  1-3).  The  pilot  was  informed  of  the 
sensitivity  of  the  GDI  setting  before  each  approach. 

The  first  of  the  seven  consecutive  approaches  was  a  practice 
run  and  was  flown  from  the  initial  approach  fix  (VIHITE  in  Figure 
1-2  or  TRIXI  in  Figure  1-3)  with  a  GDI  sensitivity  of  1/4  nm  per 
dot.  When  the  missed  approach  point  (MAPS)  was  reached,  the 
safety  pilot  instructed  the  pilot  to  execute  the  missed  approach 
procedure  and  to  return  to  INNES  (EDSUS)  for  another  approach. 

The  flight  from  the  missed  approach  point  to  INNES  or  EDSUS  could 
be  flown  without  the  hood  if  the  pilot  wished. 

The  pilot  was  reguired  to  maintain  an  air  speed  of  110  knots 
during  the  entire  approach  and  an  altitude  of  2,500  feet  until 
the  final  approach  fix.  He  could  then  descend  to  1,500  feet  as 
indicated  on  the  approach  plate. 

The  pilot  decided  when  to  initiate  each  turn  by  referencing 
the  distance-to-waypoint  readout  on  the  LORAN-G.  He  was  asked  to 
perform  standard  procedures  at  specific  points  in  the  approach 
to  maintain  workload  at  a  realistic  level,  and  to  duplicate  the 
activities  reguired  during  an  actual  instrument  approach  at  an 
uncontrolled  airport,  including; 

o  Monitoring  the  Automated  Terminal  Information  Service 
(ATIS)  frequency  of  a  nearby  airport  at  the  Initial 
Approach  Fix  (lAF)  to  obtain  the  altimeter  setting. 

o  Galling  the  Gommon  Traffic  Advisory  Frequency  (GT.4F)  to 
notify  area  traffic  of  the  approach  at  one  and  tv.’o 
minutes  inbound  from  the  Final  Approach  Fix  (FAF) . 

The  pilot  was  debriefed  at  Minute  Man  Airport  after  the 
seven  approaches  had  been  completed.  Debriefing  discussions 
included  obtaining: 

o  The  pilot's  perception  of  GDI  sensitivity's  influence 
on  workload. 

o  The  percentage  of  time  on  the  approach  that  the  pilot 
spent  monitoring  the  GDI  and  the  directional  gyro. 
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2. 


RESULTS 


2.1  GDI  SENSITIVITY'S  INFLUENCE  ON  CROSSTRACK  ERROR 

The  primary  measure  of  pilot  performance  was  crosstrack 
error.  The  crosstrack  error  was  sampled  by  the  minicomputer  once 
per  second  from  the  LORAN  receiver.  The  sampled  crosstrack  error 
was  stored  together  with  the  distance  of  the  airplane  from  the 
MAP  at  the  time  of  the  sample. 

This  positional  information  is  represented  in  Figure  2-1. 

The  matrix  in  Figure  2-2  shows  how  the  crosstrack  error  data  in 
Figure  2-1  is  organized.  The  dashed  line  in  the  center  of  Figure 
2-2  represents  the  centerline  of  the  approach  course,  and  the  MAP 
is  represented  by  a  zero  on  the  ordinate  at  the  left  of  the 
figure.  The  numbers  above  the  zero  on  the  ordinate  represent 
distances  in  nautical  miles  from  the  MAP. 

The  three  rows,  labeled  1,  2,  and  3,  are  called  "windov;s"  in 
this  report  and  are  defined  by  the  four  distances  indicated  on 
the  left.  For  example,  Window  1  is  between  three  and  five 
nautical  miles  from  the  missed  approach  point.  The  seven  columns 
in  the  matrix  are  called  "zones."  The  numbers  on  the  abscissa 
indicate  the  centerline  in  the  zones  and  correspond  to  the 
numbers  on  the  abscissa  of  Figure  2-1. 

Figure  2-1  shows  the  crosstrack  error  performance  of  the 
twelve  pilots  as  a  function  of  window  and  CDI  sensitivity.  Each 
data  point  represents  the  percentage  of  time  that  the  pilots 
spent  in  that  zone  for  the  window  and  sensitivity  indicated.  The 
percentage  of  time  that  each  pilot  spent  in  each  zone  was 
calculated  independently  for  each  window.  The  calculation  was 
made  from  the  crosstrack  error  data  collected  on  each  approach. 
Summed  across  zones,  the  seven  percentages  represented  for  a 
particular  sensitivity  add  up  to  100%  for  each  window.  Each  data 
point  represents  data  from  approximately  24  approaches. 

Figure  2-1  also  indicates  that,  as  pilots  progressed  from 
Window  1  to  Window  3,  they  spent  more  time  in  the  more  central 
zones,  and  that  the  time  spent  in  the  more  central  zones 
increased  with  CDI  sensitivity.  The  curves  representing  the 
three  lower  sensitivities  are  skewed  to  the  right.  Pilot 
performance  using  the  1/32-mile  and  1/64-mile  sensitivities  is 
good  immediately  following  the  intercept  and  stays  good  or 
improves  as  the  pilot  continues  along  the  approach  course. 
Performance  at  the  three  lower  sensitivities  starts  out  poorly  in 
Window  1  but  improves  somewhat  as  the  pilots  get  closer  to  the 
MAP. 
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FIGURE  2-1  PERCENTAGE  OF  TOTAL  PILOT  TIME  SPENT  IN  EACH  OF  SEVEN 

ZONES  FOR  EACH  OF  THREE  WINDOWS  UNDER  SIX  CONDITIONS 
OF  CDI  SENSITIVITY 


The  distribution  of  airplane  positions  across  the  zones  at 
the  lower  sensitivity  levels  appears  to  be  largely  the  result  ot 
the  pilot's  inaccurately  intercepting  the  approach  course,  and 
the  initial  overcorrections  back  to  the  left  (from  the  pilot's 
point  of  view)  to  compensate  for  such  errors.  (In  the  test 
trials,  the  approach  course  was  intercepted  from  the  left.) 

To  determine  if  tracking  performance  is  reliably  related  to 
GDI  sensitivity  and  longitudinal  position  on  the  approach  course, 
the  crosstrack  error  data  was  converted  to  root  mean  square 
values.  This  conversion  changed  negative  to  positive  scores 
(airplane  locations  to  the  left  of  the  approach  course  produced 
negative  scores)  and  amplified  the  effects  of  large  errors.  The 
resulting  RMS  scores  appear  in  Table  2-1  for  each  sensitivity  and 
the  three  windows.  The  score  in  each  cell  is  the  mean  of  the  six 
pilots'  performance. 

The  statistical  significance  of  the  differences  among  the 
data  was  examined  with  the  use  of  the  Statistical  Analysis  System 
Procedure  for  General  Linear  Models  (SAS  PROG  GLM)  computer 
package  for  statistical  analysis.  The  effects  of  window  and 
sensitivity  were  statistically  significant  (F  =  10  26, 
df  =  2,  p  <0.01)  and  (F  =  152.21,  df  =  1,  p  <0.01),  respectively. 
Also  significant  was  the  window  by  sensitivity  interaction 
(F  =  3.57,  df  =  2 ,  p  <0.05). 

Table  2-1  illustrates  these  three  effects: 

o  The  size  of  the  error  decreases  as  the  pilot  flies  from 
Window  1  to  Window  3. 

o  The  size  of  the  error  decreases  as  sensitivity  is 
increased  from  1/2  to  1/64  mile  per  dot. 

o  The  higher  degrees  of  accuracy  that  resulted  from 

increases  in  sensitivity  were  greater  for  Windows  1  and 
2  than  they  were  for  Window  3.  This  indicates  that, 
with  higher  sensitivity  levels,  pilots  were  quicker  in 
establishing  the  airplane  on  the  approach  course,  and 
they  were  more  accurate  in  flying  the  course. 

Typical  approach  path  tracks  of  pilots  flying  a  very 
sensitive  (1/64)  and  a  very  insensitive  (1/2)  needle  appear  in 
Figure  2-3.  Notice  that  pilots  have  smaller  maximum  deviations 
from  the  centerline  and  make  more  centerline  crossings  when 
fjying  with  the  more  sensitive  GDIs.  Genter  crossings  require  a 
heading  change  to  get  back  on  centerline  and  thus  are  a  source  of 
increased  workload. 
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TABLE  2-1 


RMS  CROSSTRACK  ERROR  IN  NAUTICAL  MILES  FOR  APPROACH 
WINDOW  CDI  SENSITIVITY 


CDI  Sensitivity 

Approach 


Window 

1/2 

1/4 

1/8 

1/16 

1/32 

1/64 

1C 

I 

0.26 

0.15 

0.14 

0.09 

0.06 

0.06 

0.13 

II 

0.23 

0.11 

0.08 

0.05 

0.03 

0.03 

0.09 

III 

0.14 

0.08 

0.08 

0.05 

0.05 

0.03 

0.08 

1C 

0.21 

0.11 

0.10 

0.07 

0.05 

0.04 

0.10 
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FIGURE  2-3. 


TYPICAL  APPROACH  PATH  TRACKS  FLOWN  WITH  SENSITIVE 
(1/64)  AND  INSENSITIVE  (1/2)  CDI  SETTINGS 


Figure  2-4  illustrates  the  course  width  required  to  contain 
95%  of  the  maximum  course  deviations  to  be  expected  with  each  of 
the  six  CDI  sensitivity  levels  in  each  of  the  three  windows.  The 
limits  of  each  of  the  course  widths  are  represented  by  the  18 
horizontal  lines.  The  limits  of  these  lines  v/ere  derived  from 
the  crosstrack  performance  measured  on  approximately  24 
approaches.  (In  some  cases,  only  22  approaches  were  available 
for  calculation  due  to  missing  data.)  In  order  to  determine  the 
limits  of  these  lines,  standard  deviations  of  the  maximum 
distances  traveled  from  the  centerline  on  each  of  the  22  to  24 
runs  were  calculated.  The  limits  of  the  horizontal  lines 
illustrating  course  width  represent  the  mean  of  the  24  maximums 
plus  two  standard  deviations  from  the  centerline.  The  lines  are 
asymmetrical  because  the  calculations  were  done  independently  for 
excursions  to  each  side  of  the  centerline. 

This  figure  is  another  way  of  showing  the  same  relations 
illustrated  in  Figure  2-1,  but  it  presents  the  data  in  a  way  that 
is  easier  to  relate  to  requirements  for  approach  course  width. 
Again,  the  higher  sensitivity  levels  produce  narrower  course 
width  requirements.  For  example,  at  a  distance  of  1.5  to  3  miles 
out  from  the  threshold,  the  approach  course  must  be  0.63  of  a 
nautical  mile  to  include  approximately  95%  of  all  maximum 
excursions  from  the  centerline  if  a  1/4-mile  CDI  sensitivity  v;ere 
used,  whereas  approximately  1/2  that  width  (0.32  of  a  nautical 
mile)  would  be  required  if  the  1/16-mile  sensitivity  level  were 
used . 


2 . 2  SEQUENCE  EFFECTS 

Several  pilots  reported  that  their  flying  became  easier  as 
they  became  more  familiar  with  the  approach  procedures;  however, 
toward  the  end  of  the  seven  consecutive  approaches  they  began  to 
tire.  Since  the  test  conditions  were  counterbalanced  among 
pilots,  these  sequences  would  not  be  expected  to  influence  the 
pattern  of  test  results  that  we  obtained.  However,  we  were 
curious  about  the  influence  of  the  long  test  sessions  on  flying 
performance . 

Figure  2-5  illustrates  the  RMS  crosstrack  error  scores  for 
each  of  the  six  daily  test  runs  averaged  across  all  other  test 
conditions.  Performance  was  worst  on  the  first  trial  and  was 
best  during  the  second,  third,  and  fourth  trials.  Performance 
also  tended  to  tail  off  after  the  fourth  trial,  confirming  the 
pilots'  comments  about  getting  tired. 

Furthermore,  performance  was  never  as  good  on  the  odd  trials 
as  it  was  on  the  following  even  ones.  This  seemed  to  be  the 
result  of  practice.  The  pilots  had  their  first  experience  with  a 
new  sensitivity  level  on  the  odd  trials,  and  their  second 
experience  on  the  even  trials. 
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FIGURE  2-4.  COURSE  WIDTH  EXPECTED  TO  CONTAIN  95  PERCENT  OF  MAXIMUM 
DEVIATIONS  FROM  CENTERLINE  AS  A  FUNCTION  OF  WINDOW  AND 
CDI  SENSITIVITY 


Trial  Number 


NmI  Si.nf  t'S  ar  ♦'  <1  Vt'f  dt.  f  I  pilots  t  i  j  j  y  d».,I  •  !  >.  i  .  ■ :  >*  1 1  f 

FIGURE  2  5  RMS  CROSSTRACK  ERROR  FOR  EACH  CONSECUTIVE  DAILY  TEST 


A  multi-regression  (SAS  PROC  GLM)  analysis  of  the  data 
revealed  that  the  difference  between  odd  and  even  trials  was 
statistically  significant  (F  =  7.27,  df  =  1,  p  <0.01).  Mo 
significant  interaction  was  found  between  the  odd/even  approaches 
and  sensitivity. 


2.3  THE  COST  OF  HIGHER  APPROACH  PRECISION:  VISUAL  SCAN 

During  the  debriefing,  each  pilot  was  asked  to  estimate  th.e 
percentage  of  time  spent  monitoring  the  CDI  and  the  directional 
gyro  (DG)  during  each  of  the  three  pairs  of  approaches  of  the 
day . 


Table  2-2  shows  the  pilot  estimates  of  time  spent 
monitoring  the  DG  and  course  deviation  (CD)  for  each  of  the  six 
sensitivity  levels.  Each  of  the  twelve  averages  shown  in  the 
body  of  the  table  is  the  mean  of  approximately  twelve  estimates. 

The  table  shows  three  particu'’ a"  l'  i.nteresting  influences  of 
CDI  sensitivity  on  the  pilots'  reported  distribution  of 
attention.  At  lower  sensj.Lrvity  levels,  they  spent  more  than 
twice  as  much  time  watching  the  DG  as  they  did  the  CDI.  But  at 
the  highest  levels  that  rela  1 '  c  n<h  •' p  was  reversed,  indicating 
that  the  higher  levels  of  sensitivity  caused  them  to  "fly  the 
needle"  rather  than  a  heading. 

Pilots  spent  more  time  monitoring  the  CDI  ai  sensitivity 
increased.  Consequently,  they  spent  less  time  monitoring  the  DG 
and  other  instruments  in  the  airplane.  The  biggest  jumps  in  the 
monitoring  time  for  the  CDI  were  between  the  1/4  and  1/8  and  the 
1/16  and  1/32  sensitivity  levels.  Data  indicates  that  the  ideal 
CDI  sensitivity  for  instrument  approaches  is  somewhere  between 
1/4  and  1/32  mile  per  dot.  These  shifts  of  attention  are 
potentially  important  indicators  of  pilot  workload  and  should  be 
verified  using  measures  that  are  more  objective  than  pilot 
opinion . 


2.4  THE  COST  OF  HIGHER  APPROACH 

Each  pilot  made  an  estimate 
window  of  the  approach. 

Table  2-3  shows  the  average 
each  sensitivity  level  in  each  w 
is  the  mean  of  19  to  24  workload 


PRECISION:  WORKLOAD 

of  his  workload  during  CLtch 

of  pilot  workload  estimates  tor 
ndow.  The  number  in  eac.h  ce!  1 
estimates . 
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TABLE  2-2^ 


PILOT  ESTIMATES  OE  PERCENTAGE  OF  APPROACH  TIME  SPENT 
MONITORING  THE  CD  AND  DG  FOR  THE  SIX  LEVELS  OF  CDI 
SENSITIVITY 


CDI  Sensitivity 


Flight  Instrument 

1/2 

1/4 

1/8 

1/16 

1/32 

1/64 

CDI 

15 

17 

28 

28 

43 

47 

Gyro 

37 

33 

31 

30 

29 

21 

Total 

52 

50 

59 

58 

72 

68 

TABLE  2-3  PILOT  ESTIMATES  OF  WORKLOAD  OBTAINED  IN  THE  THREE 

APPROACH  WINDOWS  FOR  THE  SIX  LEVELS  OF  CDI  SENSITIVITY 


CDI  Sensitivity 


Window 

1/2 

1/4 

1/8 

1/16 

1/32 

1/64 

X 

2.3 

3.1 

3.1 

3.8 

4.3 

5.5 

3.7 

II 

2.5 

3.2 

3.0 

4.0 

4.2 

5.6 

3.8 

ill 

2.5 

3.3 

3.2 

4.1 

4.3 

5.6 

3.8 

X 

2.4 

3.2 

3.1 

3.8 

4.3 

5.6 

3.8 
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Workload  was  judged  to  increase  with  increases  in  GDI 
sensitivity.  It  was  approximately  the  same  for  each  of  the  three 
windows.  An  analysis  of  these  data  with  Scheffee's  multiple 
comparison  procedure  (Kirk,  1982,  pp.  121-122)  revealed  that  the 
differences  in  workload  among  the  six  sensitivity  levels  were 
statistically  significant  (P  =  0.05)  for  all  except  the 
differences  between  the  1/4  and  1/8  levels  and  between  the  1/16 
and  1/32  levels.  Each  of  the  two  pilot  groups  was  exposed  to 
only  three  of  the  six  sensitivity  levels.  No  pilot  had  both  the 
1/4  and  1/8  or  1/16  and  1/32  levels.  Thus  the  pilots  thought 
increases  in  sensitivity  caused  increases  in  workload,  but  did 
not  think  workload  varied  from  window  to  window. 

At  the  completion  of  each  approach,  the  safety  pilot 
estimated  on  a  scale  of  1  to  7  how  hard  the  pilot  appeared  to  be 
working  during  that  approach.  Table  2-4  shows  the  average  of  the 
safety  pilot  estimates  of  pilot  effort  for  each  sensitivity 
level.  The  value  in  each  cell  is  the  mean  of  approximately 
twelve  independent  estimates.  The  table  indicates  that  pilot 
effort  was  judged  to  increase  with  increases  in  sensitivity. 
Scheffee's  test  revealed  that  the  differences  between  all  pairs 
except  1/4  and  1/8  and  1/16  and  1/33  were  statistically 
significant . 

Figure  2-6  shows  the  relation  between  pilot  workload  and 
crosstrack  error.  Clearly,  increases  in  GDI  sensitivity  cause 
systematic  decreases  in  crosstrack  error  and  an  increase  in 
workload . 

A  positive  correlation  of  0.66  (p  <0.01)  was  produced  by  a 
Pearson  product  moment  correlation  coefficient  calculated  between 
the  effort  ratings  of  the  safety  pilot  and  the  workload  estimates 
of  the  pilots. 


2.5  THE  COST  OF  HIGHER  APPROACH  PRECISION:  FLIGHT  SMOOTHNESS 

We  examined  the  statistical  significance  of  the  influences 
of  GDI  sensitivity,  pilot,  and  window  on  the  seven  objective 
performance  measures  (turn  rate  and  altitude  variation,  etc.). 

To  do  this,  we  used  the  SAS  PROG  GLM  computer  package  for 
statistical  analysis. 

The  analyses  were  performed  primarily  on  RMS  difference 
scores  of  the  objective  measures.  Difference  scores  were 
calculated  as  roughness  indicators  in  handling  the  airplane.  RMS 
transformations  were  done  to  increase  the  sensitivity  of  the 
analysis.  Second  differences  were  calculated  for  altitude  as 
indicators  of  variation  around  the  norma]  descent  path  that  was 
required  for  the  approach. 
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TABLE  2-4.  SAFETY  PILOT  ESTIMATES  OF  PILOT  EFFORT  DURING  THE 
APPROACH  FOR  SIX  LEVELS  OF  CDI  SENSITIVITY 

GDI  Sensitivity 

1/2  1/4  1/8  1/16  1/32  1/64 

2.7  3.4  3.0  4.0  3.6  5.1 
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Average  (It)  Crosstrack  Error  (NM) 


0.30 


■  Crosstrack  Error  (NM) 

B  Workload 


6 


CDI  Sensitivity 


FIGURE  2  6  CROSSTRACK  ERROR  AND  PILOT  WORKLOAD  ESTIMATES  FOR 
THE  SIX  CDI  SENSITIVITY  LEVELS 
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Average  (x)  Pilot  workload  estimates 


Table  2-5  shows  the  results  of  the  analysis  for  window  and 
sensitivity  on  flight  quality  as  follows: 

o  Window  affected  groundspeed  and  altitude  variation. 

Groundspeed  variation  was  highest  in  Window  1,  which  is 
where  the  pilots  initiated  their  descent  to  the  minimum 
descent  altitude.  Altitude  variation  was  least  in 
Window  3,  where  most  pilots  leveled  out  at  the  MDA 
prior  to  reaching  the  missed  approach  point.  This  v;as 
insignificant  except  for  crosstrack  error  (see  Table 
2-1)  . 

o  Turn  rate  differences  were  the  least  for  Window  1. 

o  Sensitivity  produced  significant  effects  in  heading  and 
roll  rate  variation.  In  both  cases,  variation  was 
highest  for  the  higher  sensitivity  levels,  probably 
because  the  higher  levels  were  associated  with  more 
centerline  crossings. 

o  No  statistically  significant  window  by  sensitivity 
interactions  were  found. 

o  Significant  (p<0.01)  differences  were  found  among 
pilots  for  all  six  measures. 

These  results  indicate  that  increases  in  GDI  sensitivity  do 
influence  flight  smoothness.  However,  other  than  as  possible 
indicators  of  pilot  workload,  the  practical  significance  of  these 
findings  is  unclear. 


2.6  FLIGHT  SMOOTHNESS  AS  A  MEASURE  OF  WORKLOAD 

Table  2-6  shows  the  results  of  multiple  regression  analyses 
(PROC  GLM;  Pcorr2)  that  were  conducted.  These  analyses  were 
conducted  between  the  measures  of  flight  smoothness  and  the 
subjective  measures  (pilot  estimates  of  workload  and  safety 
pilots'  estimates  of  pilot  effort).  The  objective  measures 
recorded  during  each  approach  are  listed  in  the  left  column  of 
the  table. 

Workload  and  pilot  effort  are  represented  as  column 
headings.  The  cells  of  the  table  contain  the  correlation  of  the 
corresponding  row  and  column  variables  and  the  level  of 
statistical  significance  of  that  correlation.  The  "Pcorr2" 
option  of  PROC  GLM  was  used  to  partial  out  the  influence  of  CDI 
sensitivity  on  these  correlations.  This  was  done  to  control  the 
fact  that  both  the  pilot  and  the  safety  pilot  knew  the  CDI 
conditions  under  which  they  were  flying  during  each  approach 
That  knowledge  might  have  influenced  their  workload  and  effort 
estimates . 
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TABLE  2-5  INFLUENCE  OF  WINDOW  AND  CDI  SENSITIVITY  ON  FLIGHT 
SMOOTHNESS 

Window  Sensitivity 

Flight  Quality 


Measure 

1 

II 

III 

1/2 

1/4 

1/8 

1/16 

1/32 

1/64 

Ground  speed/d  (kts) 

* 

0.4 

0.2 

0.2 

0.3 

0.2 

0.3 

0.4 

0.3 

0.3 

Altitude/2d  (ft) 

*56.0 

55.3 

47.6 

47.9 

53.7 

44.9 

64.6 

49.5 

58.7 

Heading/d  (deg) 

1.8 

1.8 

1.8 

* 

1.7 

1.6 

1.9 

1.8 

1.9 

2.1 

Pitch/d  (deg) 

1.2 

1.3 

1.4 

1.4 

1.3 

1.2 

1.5 

1.0 

1.6 

Roll/d  (deg) 

2.6 

2.6 

2.7 

* 

2.4 

2.2 

2.6 

2.5 

3.0 

3.1 

Turn  rate/d  (deg/sec) 

* 

2.2 

2.4 

2.6 

2.4 

2.3 

2.5 

2.2 

2.4 

2.5 

p  <  0.01 


I 
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TABLE  2  6 


PARTIAL  CORRELATION  OF  FLIGHT  QUALITY  MEASURES  WITH 
PILOT  EFFORT  AND  PILOT  WORKLOAD  ESTIMATES 


Flight  Quality 
Measure 

Pilot 

Effort 

Pilot 

Workload 

Cross  Track  Error 

4-0.06 

*-E0.19 

Ground  Speed/d 

4-0.11 

-E0.10 

Aititude/2d 

4-0.02 

-E0.11 

Heading/d 

*-E0.28 

*-E0.15 

Pitch/d 

*4-0.14 

-0.04 

Roli/d 

*4-0.29 

4-0.06 

Turn  rate/d 

*4-0.21 

4-0.06 

*p  <  0.01 
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The  table  indicates  that  heading,  pitch,  roll,  and  turn  rate 
differences  were  significantly  related  (p<0.01)  to  safety  pilots' 
effort  estimates.  Crosstrack  error,  heading,  roll,  and  turn  rate 
differences  were  significantly  related  to  workload  estimates  of 
the  pilot.  These  relationships  indicate  that  certain  measures  of 
flight  guality,  as  related  to  pitch  and  turning,  should  be 
further  examined  as  potential  measures  of  pilot  workload  on 
instrument  approaches. 

It  is  generally  accepted  that  the  best  estimate  of  pilot 
workload  is  the  pilot's  own  report.  This  is  believed  true  even 
if  (as  is  generally  believed)  the  pilot's  memory  limits,  ego,  an  1 
expectations  may  influence  such  estimates.  It  is  possible, 
however,  that  the  safety  pilot,  with  less  ego  involvement  and 
more  time  to  attend  to  details,  could  provide  more  useful 
estimates  of  pilot  workload. 

Heading,  pitch,  roll,  and  turn  rate  differences  were 
associated  significantly  with  the  safety  pilots'  estimates  of 
pilot  effort.  Only  crosstrack  error  and  heading  differences  are 
significantly  associated  with  pilot  estimates  of  workload.  This 
indicates  that  -  to  the  extent  that  workload  is  reflected  in  how 
the  pilot  handles  the  airplane  -  an  observer  may  be  a  better 
judge  of  pilot  workload  than  the  pilot  himself. 


2.7  CONCLUSION 

Tables  2-1,  2-2,  2-3,  2-4,  and  2-5  indicate  that  the 
increase  in  CDI  sensitivity  resulted  in  more  accurate  flying  of 
the  final  approach  course.  This  accuracy  was  accomplished  at  the 
cost  of  a  narrowing  of  visual  scan  in  the  cockpit,  increased 
pilot  workload,  and  some  decrease  in  flight  smoothness. 

Further,  observations  by  a  cockpit  observer  of  pilot 
behavior  during  instrument  approaches  appeared  to  be  a  useful 
source  of  information  on  pilot  workload. 
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3. 


SUMMARY  AND  DISCUSSION 


Recordings  from  the  LORAN-C  indicated  that: 

o  Crosstrack  error  decreased  systematically  with 
increases  in  GDI  sensitivity. 

o  Pilot  estimates  of  workload,  instrument  scan,  and  CDI 
"flyability"  indicated  that  the  workload  increased  and 
the  extent  of  instrument  scan  decreased  with  increases 
in  CDI  sensitivity. 

o  CDI  sensitivity  did  cause  a  significant  change  in  the 
quality  of  the  pilot's  handling  of  the  airplane. 

Pilot  and  safety  pilot  estimates  of  workload  were 
significantly  correlated.  ilowever,  the  estimates  of 
the  safety  pilot  appeared  more  highly  correlated  -with 
objective  measures  of  flight,  quality,  such  as  varia¬ 
tions  in  pitch,  roll,  and  turn  rate,  than  were  the 
pilot  estimates. 

Our  study  indicates  that  for  cert.ain  instrument  approach 
conditions,  a  more  sensitive  CDI  than  the  standard  currently  used 
may  be  advantageous.  For  these  airpoits,  which  cannot  have  wide 
approach  courses  due  to  terrain,  a  CD]  sensitivity  of  1  1/4  mile 
off  course  for  full-scale  deflection  is  most  often  used.  This 
sensitivity  is  also  recommended  by  the;  Radio  Technical  Commission 
for  Aeronautics  (RTCA)  for  LORAN-C  ins;trumont  approaches. 

However,  we  found  that  using  a  more  se^nsitive  needle,  v.'hich 
deflected  to  full  scale  when  only  one-quarter  of  this  distance 
off  course  (1,900  ft.),  would  decrease-  crosstrack  error  by  40%  to 
30%  . 


Crosstrack  error  accounts  for  a  major  proportion  of  system 
error  budget  used  by  procedure  design  specialists  for  designing 
LORAN-C  instrument  approaches.  Reducing  the  value  of  the 
crosstrack  error  (flight  technical  error)  component  of  this 
budget  could  narrow  the  path  that  needs  to  be  cleared  for  LORAN-C 
approaches,  thus  making  such  approachf;s  possible  in  locations 
where  currently  they  are  not. 

The  problem  with  an  increase  in  CDI  sensitivity  is 
uncertainty  about  the  workload  that  m^ly  accompany  it.  Our 
results  indicate  that  althougth  a  meas'.U’"able  increase  in  workload 
is  likely  with  such  an  increase,  large-  changes  in  pilot  control 
of  the  aircraft  would  not  occur. 
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4. 


RECOMMENDATIONS 


Our  results  indicate  that  further  research  is  needed.  A 
reliable  and  valid  means  of  measuring  pilot  workload  during 
instrument  approaches  needs  to  be  developed.  Present  research 
indicates  three  potentially  fruitful  avenues  for  further  study 
that  involves  judgment  of  an  observer  and  measurement  of  visual 
scan: 

o  Much  time  and  effort  have  been  spent  on  developing 

procedures  for  structuring  pilot  judgment  for  better 
estimating  pilot  workload.  Similar  efforts  concerning 
the  judgment  of  an  observer  in  the  right  seat  should  be 
taken . 

o  Research  is  needed  on  the  impact  of  cockpit  workload  on 
pilot  visual  scan.  Changes  in  visual  scan  could 
provide  objective  measure  of  workload  that  has  both 
operat’, al  and  face  validity.  Techniques  should  be 
develr_,c:d  for  measuring  the  pilot's  visual  scan  in  the 
cockpit.  The  more  difficult  the  task  is,  the  more 
'^■-sual  scan  is  reduced.  This  phenomenon  has  been 
demonstrated  in  the  laboratory  and  in  automobile  tests 
on  the  highways.  However,  s-.uch  demonstrations  use 
equipment  and  data  analysis  procedures  that  are 
impractical  for  cockpit  use. 

o  Logically,  workload  would  influence  how  the  pilot 

handles  his  aircraft,  and  we  have  some  data  to  support 
this  notion.  This  relations-.hip  should  be  researched 
directly  by  examining  the  influence  of  workload  on 
pilots'  use  of  the  yoke. 
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ACRONYMS 


A 

ATIS  Automated  Terminal  Information  Service 

B 


C 

CD 

CDI 

CTAF 

D 

DG 

E 

F 

FAF 

FTE 

G 

H 

I 


lAF 

IFR 

ILS 


J 

K 

L 


Course  Deviation 

Course  Deviation  Indicator 

Common  Traffic  Advisory  Frequency 


Directional  Gyro 


Final  Approach  Fix 
Flight  Technical  Error 


Initial  Approach  Fix 
Instrument  Flight  Rule 
Instrument  Landing  System 


LORAN  Long-range  Navigation 

M 

N 

NM  Nautical  Mile 


O 

P 

Q 

R 


RMS  Root  Mean  Square 

RTCA  Radio  Technical  Commission  lor  Aeronautics 
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<  G  H 


S 

SAS  PROC  GLM  Statistical  Analysis  System  Procedure  for  General 
Linear  Models 

WXYZ 
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